The present invention generally relates to distributed file systems, and more particularly to distribution of physical file systems using a shared data coherency and lock control service.
Conceptually, a physical file system architecture includes system meta-data, Msys, object meta-data, Mobject, and object data. The Msys data describes the entire file system. For example, Msys includes system parameters, an i-node bitmap, a block bitmap and other file system-dependent information.
The Mobject data describes a file or directory. In an example physical file systems, a particular Mobject segment includes the file i-node and direct and indirect blocks. The i-node includes information such as the name of the file, access rights, update times, pointers to data blocks, and pointers to indirect, double indirect, and triple indirect blocks. The indirect blocks also contain pointers to data blocks. The file data is the data referenced and manipulated by one or more client application programs.
A distributed physical file system is generally characterized by multiple storage servers that store and serve data to client applications. The storage servers and systems that host the client applications are connected to a network. The physical file system (PFS) generally provides a locking mechanism to enforce exclusive access to some of the storage space. Specifically, the PFS coordinates locking the Msys and Mobject data. For example, when a client is writing to a file, the i-node is locked to prevent another client from manipulating the same file. Any operations that result in allocating or de-allocating i-node or data blocks cause the associated bitmap areas to be locked.
In order to provide acceptable performance, distributed PFSs also support local caching of file data at the systems that host the client applications. However, in order to support caching, the distributed PFS includes coherence control logic to ensure that the client applications share a consistent view of the file data.
Many distributed PFSs are proprietary, making it difficult to spread data in present file systems to distributed file systems. In addition, present distributed PFSs, for example, Frangipani, GFS, and FPFS offer a distributed lock manager and service at the kernel, which may not scale well beyond a certain number of storage servers.
A system and method that address the aforementioned problems, as well as other related problems, are therefore desirable.
In various embodiments, the invention provides client applications with access to file data in a distributed file system. Each of the client applications is coupled to a virtual file system that that provides an interface to one or more physical file systems. The virtual file system interfaces with a lock agent that runs locally with the virtual file system. The virtual file system is invoked when client applications request file access, and validation requests are submitted to the lock agent by the virtual file system in response to the file access requests. The lock agent in turn sends lock requests to the lock server. The lock server manages locks and data coherency for the physical file systems. When the lock server grants a lease, the lease and a validation code are returned to the lock agent. The xe2x80x9cleasexe2x80x9d is a lock that is granted for a specified period of time, and the validation code indicates whether the data in the buffer cache of the virtual file system are valid. The lock agent then returns the lease to the virtual file system, which then submits the file access request to a selected one of the physical file systems. Having the data coherency and lock service at the level of the virtual file system allows distribution of virtually any physical file system in an efficient and straightforward manner.
It will be appreciated that various other embodiments are set forth in the Detailed Description and Claims which follow.